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ABSTRACT 



With respect to the specific applications at KSC, particular emphasis is placed on the use of the 
Demos software package (Lumina Decision Systems, 1993). 
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FIGURE 10. 
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Note that in this situation the alternatives and the criteria to use for evaluating alternatives are 
not immediately obvious. Hence, this is an ill-structured problem. 

Operations Research/Management Science is basically concerned with solving well-structured 
problems. A variety of methodologies, procedures, techniques, algorithms, software packages, 
etc. have been developed to solve such problems. 
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2. ELEMENTS OF A PROBLEM AND AN ASSOCIATED SOLUTION METHODOLOGY 



An analyst is a technical expert in solving problems. As such, an analyst helps the decision 
maker in the structuring of performance measures, and the development of models for problem 
solution. 

Performance measures, objectives, goals, and attributes are all things that can be used to rank 
alternatives for problem solution. As examples, an objective represents a "direction" for 
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A model for evaluation is basically a way to map alternative solutions into the outcome space 
(i.e., attribute values). The outcome associate with the alternative may be either deterministic 
or probabilistic in nature, depending on the states of nature that exist. A wide variety of 
modeling methodologies are available for evaluation, including analytic models, simulation 
models, gaming models, and judgmental models (Miser and Quade, 1985). An example of part 



of an analytic model would be an equation which relates the cost associated with performing 





mired of ranking/optimization models and techniques: an optimization process (fear the first 
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(as a result of a good problem formulation) and has kept the decision makers heavily involved 
through the problem formulation and solution processes. 
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ATING WELL-STRUCTURED PROBLEMS FROM ILL-STRUC 
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referring to the original problem statement. 



What would be obtained following this would be a portion of a problem network as shown in 
Figure 1, where PS1 was the initial problem statement, and PS2 was an answer to the question: 
"Why is this a problem?" and PS3 and PS4 are answers to the questions "Why else?" 
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" What’s Stopping us from solving this problem (e.g., problem PS1)?" 

This question is usually asked in relation to the initial problem statement (PS1), but it could be 
asked in relation to any of the other problem statements. 

Following this process, one should have a problem statement network which would appear 
something like the one in Figure 3. 




PS5 PS6 PS7 PS8 (Why?) 



Get more well-trained inspectors. 




on the blackboard. 


Objectives and attributes for a problem typically fall into a "hierarchy". At the top of the 
hierarchy are the more general objectives, while towards the bottom of the hierarchy are the 
more specific objectives. Finally at the bottom of the hierarchy are the specific attributes, each 
one connected to a specific objective. As an example, consider the hierarchy below, reprinted 
from page 105 of French (1986). 
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The objective that might be considered here would be: 

Maximize the net present worth of expected savings. 

Hence, one would need some type of evaluation model (represented as a sequence of equations) 
which would output the net present worth of expected savings as a function of the proposed 
modification. 


The various types of models, methodologies, and techniques from operation research cannot be 
easily classified as either evaluative in nature or optimization-based in nature; often, a particular 
type of model allows the performance of both functions. However, the emphasis is usually on 
one area or the other. 
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Usually, several different statistical problem must be addressed when one is using a simulation 
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In an analytic model , mathematical statements are used to represent the relation that hold 
between the variables of interest (page 195, Miser and Quade, 1985). These models typically 
do not represent a system in as much detail as a simulation model, hence they are typically not 
as accurate as simulation models. Instead, analytic models represent a system as a group of 
(either simultaneous, or serial) equations. 
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In problem situations where the alternatives are expressed m terms of sequences of interrelated 

decisions, a decision tree may be an appropriate modeling technique. 

explicitly account for uncertain factors in the decision making process, as illustrated by Figure 

5, reprinted from page 260 of French (1986). 
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Figure 5. An Example of a Decisi 




problem, the initial decision to be made (authorize or abandon development) is dependent 
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Scorecard approaches are appropriate when there are only a few alternatives (say 10 or fewer) 
to consider, when there are several decision makers, and/or when there are many attributes, at 
least some of which may be probabilistic in nature. 
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Assessment of a MAU function is usually much more difficult than the assessment of a MAV 
function since the decision maker must answer questions about hypothetical situations involving 
multiple objectives and uncertainty/risk. See Keeney and Raiffa (1993) or French (1986) for 



further information about MAU functions. 
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an infinite number of alternatives, some type of implicit enumeration is needed. This turns out 
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A wide variety of mathematical programming models allows one to represent optimization 
situations like the ones described above. Examples of these mathematical programs include 
linear programs, integer programs, dynamic programs, goal programs, and multiobjective 
mathematical programs. See Nemhauser (1989) for further discussion of some of these models. 

A survev of the literature reveals the existence of a large number of sophisticated optimization 




different software packages for project scheduling. 
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CTERISTICS AND THE CHOICE OF APPROPRIATE 
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Assemble the decision makers as a group and, for each question regarding 
hypothetical tradeoffs, have them provide a consensus answer. 

Assess a value/utility function for each decision maker, and then rank the 
alternatives for each decision maker. 
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If the relationship(s) are complex, yet the processes which give the relationships are well- 
understood, then a simulation model may be appropriate to use as an evaluation tool. If the 
relationship(s) are well-understood and can be written as a series of equations, than an analytic 
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P^rh year, members of the Industrial Engineering Group at KSC evaluate Phase I Small Business 
Innovation Research (SBIR) program proposals. As specified in the Phase I Evaluator 
Handbook, the rankings are accomplished by scoring the proposals in each of four dinerent 

criteria: 
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determined that expansion of the hierarchy past the four criteria listed above would not be 
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where u ; (x^ represents the i* individual attribute utility function and x, is the (possibly 
probabilistic) score for the i* attribute. (See Keeney and Raiffa (1993) for discussion of additive 
utility functions and individual attribute utility functions). 


As noted by Keeney and Raiffa (1993) and others, an additive utility function is very restrictive 
with respect to the decision maker’s preference structure. However, given the restrictions 
imposed by the Evaluator Handbook (that the total score be the sum of the individual scores), 
the additive utility function was an appropriate choice. 

The utility function approach yielded two important advantages. First, probabilistic (or 
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Output from the Demos model includes expected weighted utilities over each of the four critena 
as well as expected total utility. The Demos diagram for the model is shown in Figure 6. 





Figure 6. Demos Diagram for the SBIR Proposal Ranking Model. 
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A various of analyses have been performed since 1990. Finally, in 1995 a group from the 
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The diagram corresponding to the Demos model for this application is shown in Figure 7. 

An arc pointing from one node to a second node in the diagram indicates a dependence of the 
second node’s variable on the first node’s variable. Each node has a specific definition, which 
defines its relationship to other variables. For example, for the variable NPSAVINGS (expected 
Yearly savings from door removals done off the pud), this definition is given by. 
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Figure 7. Demos Diagram for the OMS Heater Mod Benefits Analysis Model. 






NPDRFLOW is normally distributed with a mean of 4 and a standard deviation 
of 1. 

PDRFLOW is normally distributed with a mean of .5 and a standard deviation of 
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In each case, the positive sign was chosen for what would be expected from a modification. 
That is, one would expect a processing modification to result in a cost savings for each flow, 
an initial cost, and a decrease (or at least no increase) in flow time. 

Typically, one would expect a processing modification to result in some type of change to 
existing procedures. Therefore, important variables to consider would be. 


NOCCPF = the number of occurrences of the change per flow (e.g., the number of 
times doors were not required to be removed per flow). 
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this data gathering process are illustrated in Figures 
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work in the following fashion. 



Suppose that the TPS engineer is at decision point DP4 in the decision process, for a "medium 
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